Exploring Mixture-of-Experts (MoE) and similar LLM architectures that allow large models to run with lower computational power.